Papers with language transfer
Unknown Script: Impact of Script on Cross-Lingual Transfer (2024.naacl-srw)
Copied to clipboard
| Challenge: | Existing models for high-resource languages are not available for all languages, and the vast majority of the world's languages are excluded from these models. |
| Approach: | They propose to use pre-trained models to analyze the effect of the target language and its script on cross-lingual transfer. |
| Outcome: | The proposed model is based on six models pre-trained on NER and POS tasks in the original script and romanized version. |
Limited-Resource Adapters Are Regularizers, Not Linguists (2025.acl-short)
Copied to clipboard
Marcell Fekete, Nathaniel Romney Robinson, Ernests Lavrinovics, Djeride Jean-Baptiste, Raj Dabre, Johannes Bjerva, Heather Lent
| Challenge: | Existing studies show that cross-lingual transfer from high-resource languages is promising for low-resourced machine translation. |
| Approach: | They propose to use adapter souping and cross-attention fine-tuning to leverage language transfer for Creoles, an under-served group of low-resource languages. |
| Outcome: | The proposed method improves performance over baselines but not meaningfully with adapters. |
Analyzing Bayesian Crosslingual Transfer in Topic Models (N19-1)
Copied to clipboard
| Challenge: | a theoretical analysis of crosslingual transfer in probabilistic topic models is presented . we use Gibbs sampling to quantify the loss of knowledge across languages . |
| Approach: | They propose a method to quantify the loss of knowledge across languages during crosslingual transfer in probabilistic topic models. |
| Outcome: | The proposed model quantifies the loss of knowledge across languages during this process . it is validated on a diverse set of five languages and discusses best practices for data collection and model design . |
Negative language transfer in learner English: A new dataset (2021.naacl-main)
Copied to clipboard
| Challenge: | This dataset contains annotated error causes for learner writing errors that tie learner mistakes to structures from their first language. |
| Approach: | They propose a learner English dataset enhanced with annotated error causes and concrete examples of learner errors that relate to their first languages. |
| Outcome: | The proposed dataset will be used to analyze learner errors related to language transfer from the learners’ first language. |
Tokenizer-Aware Cross-Lingual Adaptation of Decoder-Only LLMs through Embedding Relearning and Swapping (2026.eacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been primarily focused on English, leaving the multilingual ability unexplored. |
| Approach: | They propose a technique that creates new tokenizers and tunes embeddings on fixed model weights for target language adaptation. |
| Outcome: | The proposed method is light-weight and performant but has limitations for older models and high resource languages. |
Syn2Vec: Synset Colexification Graphs for Lexical Semantic Similarity (2022.naacl-main)
Copied to clipboard
| Challenge: | In this paper we examine patterns of colexification as an aspect of lexical-semantic organization, and compare several approaches to build large scale graphs across 499 world languages. |
| Approach: | They propose to use patterns of colexification as an aspect of lexical-semantic organization to build large scale synset graphs across a typologically diverse set of 499 world languages. |
| Outcome: | The proposed models are evaluated against human judgments on a semantic similarity task for nine languages. |
Code-Switching Curriculum Learning for Multilingual Transfer in LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit near human-level performance in various tasks, but performance drops after a handful of high-resource languages due to the imbalance in pre-training data. |
| Approach: | They propose a code-switching curriculum learning model to enhance cross-lingual transfer for LLMs by progressively training models with a curriculum consisting of token-level code-changing, sentence-level codeswitching, and monolingual corpora. |
| Outcome: | The proposed model improves language transfer to Korean, with significant gains in Japanese and Indonesian . the proposed model mitigates spurious correlations between language resources and safety alignment . |
Epitran: Precision G2P for Many Languages (L18-1)
Copied to clipboard
| Challenge: | Epitran is a multilingual, multi-back-end system for grapheme-to-phoneme transduction . it supports 61 languages and is open source under an MIT license . |
| Approach: | Epitran is a multilingual back-end system for grapheme-to-phoneme transduction . it takes word tokens in the orthography of a language and outputs a phonemic representation . Epitran's efficacy has been demonstrated in multiple research projects . |
| Outcome: | Epitran is a multilingual, multi-backend system for grapheme-to-phoneme transduction . it supports 61 languages and is open source under MIT license . |
Language Adaptation of Large Language Models: An Empirical Study on LLaMA2 (2025.coling-main)
Copied to clipboard
| Challenge: | Popularity of Large Language Models (LLMs) has seen a skyrocketing increase in recent years. |
| Approach: | They present a systematic review of the language adaptation process for Large Language Models including vocabulary expansion, continued pre-training, and instruction fine-tuning. |
| Outcome: | The proposed model is based on empirical studies conducted on LLaMA2 and discussions on various settings affecting the model's capabilities. |
Soft Language Prompts for Language Transfer (2025.naacl-long)
Copied to clipboard
| Challenge: | Cross-lingual knowledge transfer, especially between high- and low-resource languages, remains challenging in natural language processing. |
| Approach: | They propose to combine language-specific adapters and soft prompts to enhance cross-lingual transfer by parameter-efficient fine-tuning methods. |
| Outcome: | The proposed methods outperform language adapters and soft prompts in 16 languages and 10 low-resource languages. |
XHate-999: Analyzing and Detecting Abusive Language Across Domains and Languages (2020.coling-main)
Copied to clipboard
| Challenge: | XHate-999 is a multi-domain and multilingual evaluation data set for abusive language detection . we show that domain- and language-adaption can lead to substantially improved abusive language detecting in the target language . |
| Approach: | They propose a multi-domain and multilingual evaluation data set for abusive language detection that allows for disentanglement of domain transfer and language transfer effects. |
| Outcome: | The proposed model can significantly improve abusive language detection in the target language in the zero-shot transfer setups. |
A Corpus for Multilingual Document Classification in Eight Languages (L18-1)
Copied to clipboard
| Challenge: | a subset of the Reuters corpus volume 2 is used to evaluate cross-lingual document classification . current best practice is to evaluate document classification on resources in one language and transfer it to another without additional resources. |
| Approach: | They propose to use a subset of the Reuters corpus to evaluate cross-lingual document classification . they propose to add Italian, Russian, Japanese and Chinese to the subset . |
| Outcome: | The proposed subset of the Reuters corpus has balanced class priors for eight languages. |
Mitigating Language Confusion through Inference-time Intervention (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods to address the problem of language confusion are incontext learning and supervised fine-tuning (SFT) however, they consume context window space and require extensive data collection. |
| Approach: | They propose a language-sensitive intervention that detects and assesses language confusion without additional complex mechanisms. |
| Outcome: | The proposed method detects language confusion and assesses content quality without additional complex mechanisms. |
Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-Training (2025.findings-acl)
Copied to clipboard
Zhijun Wang, Jiahuan Li, Hao Zhou, Rongxiang Weng, Jingang Wang, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Shujian Huang
| Challenge: | Large language models (LLMs) exhibit remarkable multilingual capabilities despite the extreme language imbalance in the pre-training data. |
| Approach: | They investigate the existence of code-switching in the pre-training corpus and categorize it into four types within two quadrants. |
| Outcome: | The proposed approach improves performance across benchmarks and representation space. |
Meta-Learning for Fast Cross-Lingual Adaptation in Dependency Parsing (2022.acl-long)
Copied to clipboard
Anna Langedijk, Verna Dankers, Phillip Lippe, Sander Bos, Bryan Cardenas Guevara, Helen Yannakoudakis, Ekaterina Shutova
| Challenge: | Meta-learning can help overcome resource scarcity in cross-lingual NLP problems . pre-training of models requires large annotated training sets for the task at hand . |
| Approach: | They propose to use meta-learning to train a model to learn a parameter initialization that can adapt quickly to new languages. |
| Outcome: | The proposed model-agnostic meta-learning improves on language transfer and standard supervised learning baselines for unseen, typologically diverse, and low-resource languages in a few-shot learning setup. |
Tracing L1 Interference in English Learner Writing: A Longitudinal Corpus with Error Annotations (2025.emnlp-main)
Copied to clipboard
Poorvi Acharya, J. Elizabeth Liebl, Dhiman Goswami, Kai North, Marcos Zampieri, Antonios Anastasopoulos
| Challenge: | high-quality learner corpora are rarely available for studies of second language acquisition and language transfer. |
| Approach: | They propose to curate a corpus of adult learners with longitudinal data that includes 15 different L1s. |
| Outcome: | The proposed corpus contains 687 texts written by adult learners in the USA . authors show that the corpus can be used to explore language learning trajectories over time. |
Data-Centric Continual Pre-training for 500+ Languages: A New Bilingual Translation Corpus and Multilingual Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models pre-trained on massive data have promoted multilingual natural language processing (NLP). |
| Approach: | They construct a bilingual translation corpus with 2,500 language pairs and develop a suite of four models with parallel data. |
| Outcome: | The proposed model suites are evaluated across 7 tasks and 12 benchmarks. |
Improving Language and Modality Transfer in Translation by Character-level Modeling (2025.acl-long)
Copied to clipboard
| Challenge: | Current translation systems cover only 5% of the world's languages . expanding to the long-tail of low-resource languages requires data-efficient methods that rely on cross-lingual and cross-modal knowledge transfer. |
| Approach: | They propose a character-based approach to improve adaptability to new languages and modalities by using a teacher-student approach and parallel translation data to obtain a SONAR character-level encoder. |
| Outcome: | The proposed model outperforms subword-based models in speech-to-text translation on the FLEURS benchmark on 33 languages and achieves state-of-the-art generalizability to unseen languages. |
RomanLens: The Role Of Latent Romanization In Multilinguality In LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit strong multilingual performance despite training on English-centric corpora. |
| Approach: | They propose to use Romanization as a potential bridge in multilingual processing . they propose to encode semantic concepts similarly across native and Romanized scripts . |
| Outcome: | The proposed model encodes semantic concepts across native and Romanized scripts, suggesting a shared underlying representation. |
Towards a Danish Semantic Reasoning Benchmark - Compiled from Lexical-Semantic Resources for Assessing Selected Language Understanding Capabilities of Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | a semantic reasoning benchmark for Danish is compiled from human-curated lexical-semantic resources. |
| Approach: | They present a semantic reasoning benchmark for Danish compiled semi-automatically from a number of human-curated lexical-semantic resources. |
| Outcome: | The proposed datasets are compiled semi-automatically from human-curated lexical-semantic resources. |
Mixture of Languages: Improved Multilingual Encoders Through Language Grouping (2025.emnlp-main)
Copied to clipboard
João Maria Janeiro, Belen Alastruey, Francisco Massa, Maha Elbayad, Benjamin Piwowarski, Patrick Gallinari, Loic Barrault
| Challenge: | Recent work in this field relies on training transformer encoders on a large amount of multilingual data, with all parameters shared across all languages. |
| Approach: | They propose a mixture of languages strategy to pretrain largely multilingual encoders using masked language modeling. |
| Outcome: | The proposed architecture outperforms a dense counterpart, MoE models and public multilingual encoders on downstream tasks while minimizing interference. |